Interventions to promote cost-effectiveness in adult intensive care units: consensus statement and considerations for best practice from a multidisciplinary and multinational eDelphi study

Background There is limited evidence to guide interventions that promote cost-effectiveness in adult intensive care units (ICU). The aim of this consensus statement is to identify globally applicable interventions for best ICU practice and provide guidance for judicious use of resources. Methods A three-round modified online Delphi process, using a web-based platform, sought consensus from 61 multidisciplinary ICU experts (physicians, nurses, allied health, administrators) from 21 countries. Round 1 was qualitative to ascertain opinions on cost-effectiveness criteria based on four key domains of high-value healthcare (foundational elements; infrastructure fundamentals; care delivery priorities; reliability and feedback). Round 2 was qualitative and quantitative, while round 3 was quantitative to reiterate and establish criteria. Both rounds 2 and 3 utilized a five-point Likert scale for voting. Consensus was considered when > 70% of the experts voted for a proposed intervention. Thereafter, the steering committee endorsed interventions that were identified as ‘critical’ by more than 50% of steering committee members. These interventions and experts’ comments were summarized as final considerations for best practice. Results At the conclusion of round 3, consensus was obtained on 50 best practice considerations for cost-effectiveness in adult ICU. Finally, the steering committee endorsed 9 ‘critical’ best practice considerations. This included adoption of a multidisciplinary ICU model of care, focus on staff training and competency assessment, ongoing quality audits, thus ensuring high quality of critical care services whether within or outside the four walls of ICUs, implementation of a dynamic staff roster, multidisciplinary approach to implementing end-of-life care, early mobilization and promoting international consensus efforts on the Green ICU concept. Conclusions This Delphi study with international experts resulted in 9 consensus statements and best practice considerations promoting cost-effectiveness in adult ICUs. Stakeholders (government bodies, professional societies) must lead the efforts to identify locally applicable specifics while working within these best practice considerations with the available resources. Supplementary Information The online version contains supplementary material available at 10.1186/s13054-023-04766-2.


Background
There is a growing recognition among healthcare stakeholders of the responsibility of providers to deliver valuebased healthcare with the costs of care assessed alongside outcome [1,2].Intensive care treatments represent a significant proportion of increasing healthcare costs [3].To comprehensively assess the economic viability of various interventions, cost-effectiveness analysis serves as a valuable tool, examining both the costs and health outcomes associated with these interventions [4].Despite the high costs involved, there are very few cost-effectiveness studies in intensive care units (ICUs), with an average of 4.6 studies published per year [4].
Numerous policies and research efforts have aimed to improve the quality of critical care delivery, however less attention has been focused on cost-effectiveness [3].Only a few interventions have demonstrated improved clinical outcomes in critical care.Heterogeneity among critical care patients who present with varied diagnoses and require a range of different interventions also make it difficult to conduct cost-effectiveness studies focused on single items.In addition, there are no universally acceptable ways to measure ICU-related costs [5][6][7][8][9][10][11][12].Most studies describe projected benefits or estimate potential cost savings only [5].Our study aimed to go beyond cost-containment and sought to identify measures which could maximize care quality and outcomes within existing funding envelopes.Previous studies have used ICU length of stay (LOS) as a composite measure of costs and resource use [6].However, the relationship between the cost of care and the ICU LOS is imperfect [7][8][9][10].Another complicating factor is that the total ICU costs are composed of both fixed and variable costs, of which fixed costs are estimated to constitute nearly 80% of ICU costs [11,12].
The COVID-19 pandemic has changed the dynamics of ICU care significantly.Unfortunately, at the beginning of the pandemic, ICUs were unprepared for the surge of sick patients [13].Global supply-chain disruptions have prompted healthcare organizations to change their focus from 'just in time' to 'just in case' , i.e. ensuring effective critical care surge response, including consideration of ICU staffing models, having national or regional strategic reserves of personal protective equipment, devices, consumables, and pharmaceuticals [14,15].There is a need to implement cost-effective measures without compromising the safe delivery of value-based healthcare while maintaining a buffer for 'just-in-case' scenarios.Unfortunately, there is limited literature available to guide this, hence this multidisciplinary and multinational Delphi study that aimed to gather expert opinions and develop consensus to identify cost-effective interventions, the inTerventions tO Promote coSt-effectiveness In aDult intEnsive care units (TOPSIDE study).

Methods
We followed 'Guidance on Conducting and REporting DElphi Studies' (CREDES) guidelines to plan and present the Delphi study results [16].The National Healthcare Group (NHG) Domain-Specific Review Board (DSRB), Singapore approved the study with a waiver of informed consent due to the non-interventional, Delphi survey design (NHG DSRB reference number-2023/00414).Consent was implied when experts participated in the Delphi process by completing the online surveys.

Purpose and rationale
Delphi techniques are widely used in healthcare to answer relevant questions where the research is limited, ethically/logistically difficult or the evidence is equivocal [17].Previously, Delphi methods have been used in the ICU settings to reach a consensus on a standard set of ICU discharge criteria [18], to optimize clinically relevant drug-drug interactions [19], to develop a set of 'top tips' for good healthcare communication [20], to define competencies [21] or research priorities [22,23].Therefore, we sought to identify the interventions to promote costeffectiveness using a modified eDelphi methodology.

Expert panel and steering committee
Experts were identified from a convenience sample of multinational subject matter experts through peer recommendation.We used World Health Organization (WHO) endorsed categorization of countries according to Gross National Income (GNI) per capita [24].Our definition of being an expert was defined as being a clinician (doctors, nurses and allied health professionals) practising in an ICU and/or an administrator with extensive ICU clinical and healthcare service management experience, all experts had at least 5-year post-graduate experience.
As with previously published guidelines, the decision regarding the number of experts was a pragmatic choice; a key consideration was to ensure good representation must lead the efforts to identify locally applicable specifics while working within these best practice considerations with the available resources.
with qualified intensive care providers [25].Patients and the public were not involved in the study.We formed a steering committee of 13 experts to guide the study and associated discussions (AK, JML, KCS, SR, MC, CB, ACM, RDS, SR, JPF, VMN, BKR, RCM).The steering committee comprised colleagues holding current or recent leadership positions in various national and international Intensive Care societies or were senior ICU clinicians, and/or senior researchers with at least 10-year experience.Due to the size of the steering committee, and the ability to blind the committee to the results prior to completion of each round it was decided to include steering committee members in the expert group given their international leadership roles and expertise in the subject matter.One member of the steering committee, AK, administered and coordinated the Delphi survey and therefore was the only member of the steering committee not to participate in the expert group.

Description of the methods
The Delphi study was conducted from November 2022 to February 2023, followed by steering committee discussions in March-June 2023.Experts were given 3 weeks to respond to each round, with 2 reminders a week apart.
This was a modified Delphi survey with a succinct mixed qualitative and quantitative approach.The Delphi surveys were administered through an online Delphi survey platform (Welphi ™ , Decision Eyes, Lisbon Portugal), facilitating global expert involvement and streamlined collation of responses.
Round 1 consisted of an open-ended questionnaire.Experts were requested to propose interventions in four domains, following the structure set out by the US National Academy of Medicine in their approach to highvalue healthcare [26].The four domains are: In the first Delphi round, experts were invited to list and provide comments regarding interventions that promote cost-effectiveness considering resource limitations, especially in the peri-pandemic era.Individual responses were collated and converted into a list of 60 interventions for the second round.Round two of the Delphi process was composed of quantitative as well as qualitative components.All the experts from round 1 were included in round 2, irrespective of whether they had responded earlier, as the first round was deemed exploratory in nature.The experts were able to see the questionnaire, along with anonymized responses from round 1 before scoring in round 2 (Additional file 1: Fig. S1).This provided an opportunity for them to revise their judgments based on comments from their peers.In round 2 the experts were requested to rate the proposed interventions on a 5-point Likert scale (ranging from strongly agree, agree, neutral, disagree to strongly disagree) along with an opportunity to offer optional narrative comments on each statement.
At the end of round 2, qualitative and quantitative data were collated.The items from round 2 were retained for round 3, which was quantitative in nature.The third round only included experts who had responded in the second round.As before, the experts were able to see the same questionnaire from the previous round, along with anonymized responses before re-scoring (Additional file 1: Fig. S2).
We adopted a dynamic approach regarding the number of rounds, with subsequent rounds incorporated if the steering committee deemed it necessary for further clarification [27].As in previous studies, we observed that there were no changes in experts' ratings between rounds two and three [28], and the steering committee decided that another round was not warranted.At the end of round 3 a list of 50 items was produced, in order to make this appraisable the steering committee drew this into a smaller list through combining topics into common themes and identifying high-impact items.This process was undertaken through iterative rounds of discussion and voting within the steering committee.

Definition and attainment of consensus
We applied previously published consensus criteria for Delphi studies to the quantitative results of 3 rounds [25].Interventions with over 70% consensus (scored as 'strongly agree/agree'), which < 15% of respondents scored as 'strongly disagree and disagree' , were recommended as cost-effective measures.Conversely, if over 70% of experts disagreed on an intervention (by scoring strongly disagree or disagree), with fewer than 15% agreement (strongly agree and agree), this intervention was considered as 'undesirable' .An intervention was classified as achieving 'no consensus' if neither criterion was met.We also compared the list of interventions between rounds 2 and 3 to assess for impact of peer opinion among the experts.These consensus items endorsed by the Delphi experts were subjected to iterative discussions by the steering committee through face-to-face interactions, online video meetings and electronic written discussions.Each steering committee member voted to select ten interventions that they deemed important.One steering committee member (AK) was excluded from final voting due to risk of bias in view of access to identifiable comments as the lead administrator of Delphi process.Items to be identified as 'critical' required consensus by more than 50% of steering committee members.This list of 'critical' interventions formed the basis of consensus statement and best practice considerations.
A total of 60 interventions were identified at the end of round 1 and the same 60 formed the basis for rounds 2 and 3. Of the 60 interventions, 50 interventions reached consensus criteria after round 2 and the same 50 items achieved consensus in round 3 as well (Table 2).Of note, 31 (51%) of these interventions achieved a consensus of 90% and above after round three.
Ten interventions did not achieve Delphi consensus for prioritization (Table 3).
The structured details of qualitative responses from rounds 1 and 2 are presented separately (Additional file 3: Table S2).
Thereafter, the steering committee endorsed 11 interventions that were identified as 'critical' by more than 50% of steering committee members.These interventions and experts' comments were summarized as 9 final considerations for best practice (Box 1): Experts who responded in round 3 (n = 30) Physicians (n = 28, 93.3%), Nurses (n = 2, 6.7%) Low-and middle-income countries (n = 12, 40%) High-income countries (n = 18, 60%) o ICU staff should be supported in developing and maintaining critical care competency and skills to develop a continuous learning process.Development of core curriculum and structured training programmes for staff across all relevant professional groups, and mapping training onto that curriculum would further help with credentialling and ensuring standardization of ICU knowledge.o Additionally, staff competency should be regularly audited and reviewed with participation of all stakeholders.Consideration should be given for ongoing personal performance evaluation plan for each physician and end of year evaluation.
3) Development of step-down units and long-term care units: o Definition of ICU versus 'high dependency/ step-down/ long-term care units' should be established as agreed upon at regional/ national level.Consideration should be given to standard policies, good governance, and appropriately trained and skilled workforce.
4) Organization of tele-ICU services to bring down ICU costs as well as support under-served areas: o Telemedicine and remote review should be considered where access to physical presence of a trained ICU team is not possible.Any such advice should be based on a thorough assessment based on clinical, laboratory, and radiological data.A professional relationship between the remote expert and the on-site team with the ability to provide on-site visits or patient transport if required, should be considered.There should be governance and feedback process to oversee the service.

5) Adoption of dynamic staff roster to accommodate even distribution according to workload:
o ICUs should adopt a rational staffing approach introduced with flexible and even distribution of staff according to the workload, to avoid burn out and exhaustion.However, this approach should also consider the challenge of work-life balance and staff retention.

6) Implementation of end-of-life (EOL) interventions:
o Consideration should be given to multidisciplinary approach focused on patient-and family-centric EOL care, which could avoid unnecessary ICU admissions, avoid prolonged ICU stays, and challenges in withdrawing the already instituted care, thereby reducing costs through reduction in futile treatment.The specific skills and expertise required for this aspect of practice should be part of the core curriculum for intensivists and other critical care professionals.

7) Adoption of early mobilization:
o Early mobilization provided by the designated ICU physiotherapist and nursing team would help in enhancing ICU recovery in patients at the earliest opportunity and can facilitate early discharge.Frequency of laboratory and radiological tests should be optimized through quality improvement methodology.

Discussion
We conducted a multidisciplinary and multinational eDelphi survey followed up with iterative discussions among the steering committee, to gather expert consensus to identify interventions that promote cost-effectiveness in adult ICUs.Final best practice considerations included adoption of a multidisciplinary ICU model, strong focus on staff training and competency assessment, and audits to objectively promote a culture of An international consensus effort on the 'Green ICU' concept (9) Implementation of audits to promote a culture of continuous quality improvement continuous improvement.These aim to ensure high quality critical care services both inside and outside ICUs.Additionally, implementation of a dynamic staff roster, multidisciplinary approach to implementing EOL care, early mobilization and promoting international consensus efforts on the 'Green ICU' concept were endorsed.These findings represent, to the best of our knowledge, the first time a Delphi study has been used to address the complex issue of cost-effectiveness with global appeal involving experts from across varied healthcare systems, extending beyond regional and national boundaries.These best practice considerations are designed to be equally applicable to low-income as well as middle-highincome countries, albeit requiring tailoring according to local resources.Many of the proposed best practice considerations are supported by previous studies suggesting outcome benefits including actual or hypothetical cost benefits, though cost-effectiveness was not studied, e.g.case of multidisciplinary team model [30][31][32].One systematic review of cost-related impacts from utilizing respiratory therapists to deliver care showed both direct and indirect cost reductions, which were achieved through protocol utilization, specialized expertise, and autonomous decisionmaking [32].
Best practice consideration to develop step-down units, long-term care units, and tele-ICUs comes with an emphasis on ensuring appropriate staffing, adequate resources, and robust governance process.On the other hand, previously published literature is equivocal with regards to endorsing ICU-level interventions provided by other specialties (e.g.surgical intermediate care units, emergency department-based ICU) and manned by non-ICU teams, despite the availability of positive studies [5,[33][34][35][36].This hesitancy may be due to concerns regarding quality of care being provided by non-ICU teams and highlights the need to ensure standardization irrespective of clinical team involved.
Similarly, audits to promote a culture of continuous improvement [37][38][39][40], multidisciplinary team approach to implementing EOL care [41,42], and early mobilization [43] have shown cost benefits.Multidisciplinary team approach comprising physicians, clinical nurse specialists and/ or palliative team focused on improving communication with patients and patients' families at the end-of-life reduced LOS and lowered ICU costs and significantly improved nurse-assessed quality of dying [41,42].A systematic review comprising twenty-three randomized control trials involving 2308 critically ill patients showed that early mobilization decreased the incidence of ICU acquired weakness at hospital discharge, increased the number of ventilator-free days during hospitalization, and increased the discharged-to-home rate; as well as non-significant improvement in mortality (28day, ICU, and hospital).However, substantial heterogeneity among the included studies, and the low quality of the evidence, warrants caution.Our Delphi survey adds weight of expert consensus to the existing evidence.
These consensus statements hold even more importance in the post-pandemic era of 'just-in-case' , with competing priorities for the finite resources available.The WHO has recently declared an end to COVID-19 as a global health emergency in May 2023.Few of the interventions proposed by Delphi experts were categorized under pandemic preparedness (optimization of resource usage during the pandemic setting, establishment of a multidisciplinary disaster/pandemic response team and emphasis on audit to ensure compliance to best practices in a pandemic situation).As this Delphi survey was conceived and conducted during the pandemic period, experts' emphasis on pandemic preparedness is not surprising.However, these considerations are equally applicable to the non-pandemic times.
Interestingly, several of the proposed interventions did not reach consensus despite some proposed benefits in previous studies, e.g.low-cost wearables combined with artificial intelligence for physiological monitoring, pointof-care diagnostic and imaging devices, and use of disposable items over reusable [44].Possible reasons could be that such interventions need upfront expenses with unclear cost recovery downstream, and lack of confirmatory evidence of their efficacy in relevant clinical settings.
Our Delphi study has several limitations.We sought to balance the views of established experts with publication and other markers of expertise in sustainable and costeffective care with broader, lived expert experience of ICU and thus seek to ensure the views were more generalizable.While these individuals had many years of ICU experience, as a necessary pragmatic need to maintain an effective number of Delphi participants we cannot guarantee that they fully considered every issue of this admittedly broad topic, e.g.regional variations in practice due to regulatory issues surrounding ICU setups, availability of drugs and devices.Although we had a reasonable geographic (we note a lack of experts from African countries) and national income-level (excluding low-income countries) spread of experts, the opinions may not represent those from areas which are not covered.However, the experts did represent a good mix of intensive care professionals from low-middle-income, and high-income countries.
Secondly, we present percentages for agreement.However, our Delphi survey was not designed to understand the differences in case of non-agreement, any further analysis in form of a mediation analysis will not be possible given the dimensionality of our dataset.The regression models for a mediation analysis would not pass the goodness-of-fit test.
Thirdly, there was no patient involvement as well as no distinct stakeholder groups.This may have limited the ability of our findings to fully represent a holistic perspective and failure to highlight any disparate views.Our relatively small number of stakeholders and higher attrition in non-physician categories precluded such groupings (Table 1).However, even in a multiple-panel study consisting of different stakeholder groups, it is difficult to ascertain what weightings should be given to each group and there is no current guidance on this.
Additionally, our Delphi study asked the experts to categorize proposed interventions into four sub-categories, following principles of high-value health care which have previously shown promise in lowering costs [26].These principles however were not critical care specific, were derived from one high-income nation and were opinions only from senior hospital administrators.
Lastly, we had a response rate of just over 50% between round 1 and round 2 as well as higher attrition in non-physician categories.In view of the intention to achieve a global distribution of the experts, we were dependent on the peer recommendation to identify these experts.Response rates are often a challenge in studies such as this, and responders are more likely to have an interest and self-assessed expertise in the topics covered.Non-responders might have had limited interest in participating in the Delphi survey, or were unable to due to time pressure and competing priorities.However, we were able to achieve a response rate of 88% in the third round, further strengthening our view that we selected those with greatest interest and self-assessed expertise for the final recommendations.Attrition bias can occur when the participants that do not respond in subsequent rounds have different views from their peers who continue to participate.However, when we compared the round 3 and round 2 results, we observed no variation in opinions and a retention of > 80% which is considered satisfactory as per published literature [22].
While we have produced a number of best practice considerations, how they should be implemented and what metrics be used to assess their effective implementation require further work.We believe that these best practice considerations should be equally applicable to low-income as well as middle-high-income countries; albeit need tailoring according to local resources.Validation of these proposals by additional experts, involvement of more international scientific societies and involvement of patients and families are important next steps.Future research should focus on the selection of appropriately bundled interventions, tailored to regional needs and implemented under an appropriate framework of metrics.

Conclusion
We present consensus statement and best practice considerations of interventions to promote cost-effectiveness in adult ICUs.The best practice considerations include adoption of a multidisciplinary ICU model of care, focus on staff training and competency, ongoing quality audits, implementation of dynamic staff rostering, multidisciplinary approach to implementing EOL care, early mobilization and promoting international consensus efforts on Green ICU concept.Stakeholders such as government bodies and professional societies must lead the efforts to identify locally applicable specifics with the available resources, while working within these best practice considerations.

Fig. 1
Fig. 1 Flowchart to illustrate the stages of Delphi technique

( 1 ) 2 ) 3 ) 4 ) 5 )
Adoption of multidisciplinary patient care model (Development and maintenance of staff competency, and audit (Development of step-down units and long-term care units (Organization of tele-ICU services to bring down ICU costs as well as support under-served areas (Adoption of dynamic staff roster to accommodate even distribution according to workload (6) Implementation of end-of-life (EOL) interventions (7) Adoption of early mobilization

Table 2
Interventions endorsed by Delphi study, categorized into nine sub-groups #Details in supplementary table2

Table 3
Cost-effective interventions which did not achieve Delphi consensus for prioritization at this time HDU high dependency unit, ICU intensive care unit *One intervention, namely, "Integrated ICU model preferable to ED-based ICU/ Specialty-based ICUs" scored more than 70% agreement in both rounds; however, the disagreement was 15% and we counted this intervention as a "Cost-effective intervention which did not achieve Delphi consensus for prioritization"

Cost-effective interventions which did not achieve Delphi consensus for prioritization at this time Degree of endorsement (Strongly agree and agree) in round 3
Box 1 Consensus statement and considerations for best practice interventions to promote cost-effectiveness in adult intensive care units